RECONSTRUCTION OF FUNCTIONS FROM THEIR TRIPLE 

CORRELATIONS 
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Abstract. Suppose that A is a subset of an abelian group G. To know the 3-deck of A is 
to know the number of occurrences in A of translates of each possible multiset {0, a, b}. The 
concept of the 3-deck of a set is naturally extended to L 1 functions on G. In this paper we 
study when the 3-deck of a function determines the function up to translations. The method 
is to look at the Fourier Transform of the function. Our emphasis is on the real line and the 
cyclic groups. 



1. Introduction 

The aim of this article is to address a problem that arises independently in combinatorics 
and in diffraction theory. 

In combinatorics, a common problem is to reconstruct an object (up to isomorphism) from 
the collection of isomorphism classes of its sub-objects. The most famous problems of that 
nature are the Reconstruction conjecture and the Edge reconstruction conjecture and we refer 
to Bondy |Boj and Bondy-Hemminger |BH| for surveys on the subject. In that direction, 
Radcliffe and Scott investigated in |RS11 IRS2| the problem of reconstructing subsets of E, 
Z n , Q n or Z/nZ from the collection of isomorphism classes of their subsets of prescribed size. 
(Here two sets are isomorphic if they are translates of each other). 

To be more precise, fix an integer k > 3 and let G be one of the groups K, Z n , Q n or Z/nZ. 
A subset E of G is locally finite if E n (E — x) is finite for every x € G. The k-deck of E is 
then the function on G fc_1 defined by 

N E ( Xl ,...,x k - 1 ) = \e n (E - xi) n - ■ ■ n (E - x k ^)\ 

where |.| stands for cardinal. 

The problem addressed by Radcliffe and Scott is the following: 

Problem 1. Assume that E and F are two locally finite sets in an abelian group G that have 
the same k-deck, for some integer k. Are E and F translates of each other? 

It turns out that, in the particular case of G = Z/nZ, this problem also arises in the 
mathematical theory of diffraction. 
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It is well known that the answer is negative for k = 2, but for k > 4, Grunbaum and Moore 
showed that the answer is positive. This result can be further improved in some cases. For 
example it is shown in |RS1| that if n is a prime, then the answer is still positive for k = 3. 

One of our aims here is to summarize the knowledge on the problem in the Z/nZ case, to 
improve some of the proofs and to give the last possible positive results in the cases n = p a 
and n = pq with p, q primes. This bridges the results in |RSlj and |GM| . and essentially 
closes the problem. To do so, we make a more explicit use of the Fourier transform than in 
[RSI] . 

Radcliffe and Scott in |B,S2| further prove that the answer to Problem ^ is positive if the 
group is one of R, Z n , Q n . Moreover they ask for measure-theoretic counterparts to their 
results in M. This is the main question that we will address here. 

To be more precise, we will consider the Lebesgue measure on M and write again \E\ for 
the measure of a measurable subset E of R. The k-deck of E is still defined as the function 
on R fe_1 given by 

N E (xi,...,x k ^) = \e n (E - xi) n - ~ n (E - x k -i)\. 

We will also consider naturally the fe-deck of a nonnegative function / £ defined by 



N f ( Xl ,..., 




f(t)f(t + x 1 )---f{t + x k _ 1 )dt. 



If xe is the characteristic function of a set E of finite measure, then Ne = N XE . The problem 
we address is the following: 

Problem 2. Let k > 3 be an integer and let Nf denote the k-deck of the function f . 

(a) Suppose < /, g E such that Nj = N g almost everywhere. Does there exist an 
a£l such that g(x) = f(x — a) for almost every 

(b) Let E,F be two sets of finite measure such that Ne = Np almost everywhere. Does 
there exist an a £ K such that F = E — a up to a set of zero measure. 

If this happens we say that f (resp. E) is determined up to translation by its k-deck. 

Again, this problem occurred before in the context of texture analysis as well as in crys- 
tallography where the £>deck is known as an higher order autocorrelation function {see 
|AK1 ICW1 IRot j ) . In these fields, a common problem is to be able to reconstruct a func- 
tion from its 2-deck or autocorrelation function. This is easily seen to be equivalent to that 
of reconstructing a function knowing only the modulus of its Fourier Transform and is there- 
fore called the phase retrieval problem. We refer to the surveys IBM| IK^T| IMij the book Hu 
or the introduction of |.Taj for further references. Note that the 3-deck has been proposed to 
overcome the non-uniqueness in the reconstruction (see JMiJ. 

We will only give a partial answer to this question and show that in many cases, the 3-deck 
will be enough for the answer to be yes. In particular, we managed to do so for characteristic 
functions of compact sets, a fact independently proved by Rautenbach and Triesch |RT| . 
These positive results are described in Section 12.21 

In the opposite direction, for k fixed, building on a construction in |.Taj (and on an earlier 
construction in |CW| ). we construct in Section 12.31 an uncountable family of functions (fi) 
that all have same 3-deck but are not translates of one another. However, these functions 
are not characteristic functions of sets, and we have not been able to settle completely the 
second problem. Nevertheless, in Section l2~ll we have managed to prove that if / = xe and 
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g G L 1 (M) n L°°(R) is non- negative, then N g = Nf implies that g is also a characteristic 
function of some set F, thus reducing Problem |2] (a) to Problem [2] (b). However we do not 
know if F = E — a for some a£l. 

In Section \2. 51 we show that if E belongs to a rather wide class of measurable sets of finite 
measure then it is determined up to translation from its 3-deck, among all measurable sets. 
This class consists of all sets which are unions of intervals with endpoints that do not have an 
accumulation point and such that the complementary intervals have length bounded below 
by a positive constant. 

To conclude this introduction, let us point out that our investigation on the 3-deck prob- 
lem for sets establishes a link between phase retrieval problems and uncertainty principles. 
Namely, the obstruction to uniqueness in the 3-deck problem is the possibility to have large 
gaps in the spectrum (i.e. support of the Fourier Transform) of a function. Although this 
is easy to achieve for general functions, it is not known whether characteristic functions 
may have many gaps in the support of their Fourier Transform. The best results in that 
direction, due to Kargaev and Volberg |Kal IKV] . state only that the Fourier Transform of a 
characteristic function may be zero on an interval. However this is not enough for proving 
a counterexample (to determination from 3-deck) as the results of Section \l. 51 are applicable 
to sets of the type used by Kargaev and Volberg, where we prove that the specific sets used 
in |Kal IKVj are determined by there 3-deck up to translation. 

This article is organized as follows. In the Section[2J we will focus on results on the real line, 
starting with the reformulation of the problem in terms of the Fourier Transform, continuing 
with positive and negative results and concluding with the stability result mentioned above. 
In Section El we focus on the Z/nZ case, starting with a survey of known results with the aim 
of bridging the knowledge in both communities that are interested in the problem. We are 
then focusing on our own results. 

2. The results on R 

2.1. Reformulation in terms of Fourier Transforms. In this section we will reformulate 
the problem in terms of the Fourier Transform. For the the sake of simplicity of notation, we 
will only focus on the case of the real line, although the results generalize without difficulty 
to other locally compact abelian groups. 

Also the results here have been proved in a similar way in |CWj and again in |RT| . As we 
need the notations, we prefer reproducing the proofs for sake of simplicity and completeness. 

To fix notation, we define the Fourier Transform of ip G L 1 (M d ) by 

m = [ v(t)e- 2 ™^dt. 

It is easy to see that if / G L 1 (R), then N f G L 1 ^" 1 ) with {{Nf^ < [|/||*, with equality 
when / is non-negative. It is not difficult to see that actually [|iV/|| r < Y\j = i \\f\\ p . a s soon as 
1 + = Y^,j=i ^~ an d that Nf is continuous as soon as / G L X (M) n L P (R) for some p > 3. 
Actually all results for convolutions apply for fc-decks with essentially the same proofs. 
Further, the Fourier Transform of Nf is 

Nffa, e fc _i) = m) ■ ■ ■ 7<&_i)7(-(£i + • • • + • 
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This implies that solving N g = Nt is equivalent to solving 

(2.1) • • + • • • + &-0) = /&) • • • /(&-0/(-(£i + • • • + &-0) 

for all &_i £l. 

As our primary interest is in characteristic functions, we will from now on consider that 
/ and g are nonnegative L 1 functions and that /(0) := f R f(t)dt ^ 0. Another reason for 
considering only real functions is to avoid the extra complication due to the fact that the 
functions / and uf have the same /c-deck whenever u is a k-th root of unity 

In the sequel we will only consider Problem^ for non-negative functions. 

Then, taking £i = • • • = = in (|2.1|) gives <?(0) fc = f(0) k , and since g(0) and /(0) are 

nonnegative, they are equal. Further, as / and g are real /(— £) = /(£), 

It follows that, if we take £i = £, £2 = — £ and £3 = • • • = Cfc-i = in (|2.1() . we get 

l?(OI = 1/(01- 

Now, let us write 

(2.2) g(0 = e 2m ^m 

where, by the continuity of f,g, we may assume that ip is continuous on the support of /. 
Introducing this in equation (|2.1j) . it reduces to the following 



(2.3) ¥>(£i + • • • + &-0 = + ' ' ' + v(&-0 (modi) 

for all £1, ... , 6_i G supp / such that £1 + • • • + Cfc-i £ supp /. 

The solution of such an equation is then dependent on the group on which one might 
consider it. In the case of the real line, it is easy to show that the following holds (see e.g. 
jjaj, Lemma 3): 



Lemma 2.1. Suppose that f,g £ L 1 (M) are nonnegative and have Nf = N g . It follows that 
f,g are connected by (|2,2j) . Write supp f = Ij to be the decomposition of supp f into 

jc z 

disjoint open intervals numbered so that £ lo, I—j = —Ij- Then, there exists w£l and a 
sequence 9j that satisfies 9q = and 9~j = —9j such that, if x € Ij, 

if(x) = LUX + 9j. 

Moreover, if there are £1, . . . , 1 with in some Ij t such that £1 + • • • £ Ii then 

(2.4) e h + e h _ l = e l . 

In particular, if 9j 7^ 9\ then Ij and I\ are distant by at least x diam Iq . 

Sketch of proof. One first proves the result on Iq n Q where it is trivial, then extends it to all 
of Iq by continuity. 

The extension to is then obvious taking x £ 1^ , y G Iq . Finally (|2.4|) is a reformulation 
of (Q. □ 
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2.2. Positive results. Recall that a measure [i has divergent logarithmic integral if one of 
the two following integrals 

-log[/i|((-oo,x]) ^ /•"" -logHqs.oo)) ^ 



_ 1 + x 2 J 1 + x 

is divergent. 

We will now prove the following : 

Theorem 2.2. Assume that f G L 1 (M) is real valued. In each of the following cases, the 
function f is determined up to translations by its 3-deck: 

(1) f is of compact support; 

(2) for some M > the integral J \ f(x)\e M ^ dx is finite; 

(3) / is analytic in a neighborhood of the real line; 

(4) the measure f dx has divergent logarithmic integral. 

Proof. The case^is a particular case of|2]which in turn is a particular case of [21 In this cases, 
{/ = 0} is a discrete set. In the last case, a theorem of Beurling (see |Koj p. 268) states that 
/ cannot be zero on a set of positive measure. In conclusion, this theorem is proved once we 
have proved the following lemma : 

Lemma 2.3. /// does not vanish on a set of positive measure, then f is uniquely determined 
up to translations by its 3-deck. 

Following the notation of Lemma l2.11 write supp/ = suppg = Ij and let <p(x) = uox+Oj 

be the function given by the lemma. Write Iq = (—a, a). Up to changing g into g{. — u), we 
may assume that g(£) = e 27n6,J f(£) if x G Ij. 

Contrary to what we want to prove, assume that / is not almost everywhere equal to g 
and let s = inf{x > : f(x) ^ g(x)}. It follows that s is finite, otherwise / = g. 

From this we get that there is j G N with 9j ^ such that Ij is of the form Ij =]s, s + A[. 
By the last assertion of Lemma 12. II we get that 

(0, s) n {x > : + f(x) = g(x)} C (0, s - a), 

which implies that / = on (s — a, s), in contradiction to our assumption on the support of 
/■ □ 

Remark : 

(1) The proof of the lemma actually tells us that if supp/ has no gap of size diam/o 
then / is determined up to translations by its 3-deck. This, as well as Point ^ of the 
theorem has been independently proved by Rautenbach and Triesch in |RT| . 

(2) Further remarks of that order can be made. For instance, one might notice that 
diamij has to stay bounded. Indeed, if this is not the case, we may write £ G Ij as 
£ = 6 + £ - 6 with & + £ in some /,. Then 6j = <p{(-) = tpfo + f) + = 0. 

(3) As a corollary of the proof, we immediately get that if c supp / is negligible, then the 
only solution of (|2.4|) are </?(£) = ooi for some w£l 
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2.3. Negative results. 

Theorem 2.4. For every k > 3, there exist two nonnegative and smooth functions that have 
the same k-deck but are not translates of each other. 

Proof. We will only give full details of the proof in the case k = 3. 

sin(fx) N " 




Let ip(x) = ( — ^| ) , then ip G L (R) and its Fourier Transform is 

l-iei) if ici < § 

else 

Consider /(» = (l + cos(4vrx)) ip(x) so that /(£) = ^(£ + 2)+^(£) +<0(£-2). The support 
of /is then [-5,5] +{-2,0,2}. 

Now define ip to be such that ip = — 1 on [—5, 5] + {—2, 2}, tp = 1 on [— |, |] and </? = 
elsewhere. It is easy to see that <p satisfies <p(x + y) = ip(x)<p(y) whenever x,y,x + y G supp/. 

Finally, let g{x) = (l - cos(4vrx))^(x) so that </(£) = p{Qf(Q. Then iV/ = iV s but / and 
g are not translates of each other. 

To adapt the proof to the case k > 4, it is then enough to replace the cos(4-7rrr) in the 
above argument by (say) cos(2^-!-7rx). □ 

Using the same ideas as for the proof of Theorem 3 in |Jaj . one may improve this to get 
the following result: 

Proposition 2.5. For every k > 3, there exists a function f £ L 1 (IR) such that there are 
uncountably many functions, not translates of each other, that have the same k-deck as f. 

Sketch of proof The only thing to be done is to replace the factor 1 ± cos(fe + l)irx by the 
Riesz product 

Y[ (1 + a n e n cos(k + l) n irx) 

where a n decreases fast enough to ensure the convergence of the product and each e n = 
±1. ' □ 

On the other hand we cannot have this situation for all k: 

Lemma 2.6. If f and g are such that there k-decks are the same for every k, then f and g 
are translates of each other. 

This is Theorem 2 in |CWj (see also |Rotj for some generalizations). However, the proof 
bellow is a bit simpler. 

Proof. Using the notation of Lemma l2~Tl let us write g(£) = e 2m ^ +e ^ /(£) if £ G Ij. Recall 
that the 6j's satisfy, for every k, 

(2.5) n + ... + e Jk _ i = e l 

if there exists £1 G Ij 1 , • • • , £k-i G Ij h _ 1 such that £1 + • • • £fc-i G //. Write // = (ai,bi). Then, 
for k > 2 ^.^^ , there exists £1, ■ • ■ , £fc-i G Io such that £1 + • • • + £k-l G l\. Equation (|2.5|) 

then implies that B\ = so that g(£) = e 2mu ^f(^) and g is a translate of /. □ 
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2.4. A stability result for characteristic functions. In this section, we will prove that, 
under some mild restrictions, if a function has the same 3-deck as a set, then this function is 
a characteristic function of a set. More precisely, we will prove : 

Proposition 2.7. Let E CM be a set of finite measure, let f = xe and let 9 £ L 1 (M) be a 
non-negative function such that N g = Nf almost everywhere. Then there exists a set F of 
finite measure such that g = xf- 

Proof. First note that, as / = xe, \\f\\\ = II/II2 an d II/II3 = ll/ll^ 3 - Further, in this case Nf 
is continuous. 

Next, as we assumed that g > 0, we get 



AT I, 1/3 || ,r 1 1 1/3 

Ifflll = \\Ng\W = IliV/H/ 



1- 



Further, as \g\ 



we get 

IMI2 = W9W2 



\f\\ 



with Parseval. 

Now, if g £ L 3 (IR) then, by positivity of g, N g is not essentially bounded in a neighborhood 
of 0. This would contradict N g = Nf almost everywhere. 

Now that we know that g E L 3 (M), N g is also continuous and we get that N g = Nf 
everywhere. In particular, 

NI 3 = JV fl (o,o)=iv>(o,o) = ||/|| 3 . 

Finally, we have equality in the Cauchy-Schwarz inequality : 

1/2 / r \ 1/2 



/ 



9 2 = J 9 1/2 9^ < (/ 9 ) (/ . 3 ) = \\9\\l /2 \\9\\r = ll/lll /2 H/llf = 11/113 = 11^112. 



It follows that g 3 / 2 = Xg 1 ^ 2 which implies that g = Axsuppg- Finally, from J g 2 = J g we get 
A = 1 and g = Xsuppg- □ 

2.5. A restricted problem. One might think that if / and g are characteristic functions of 
sets of finite measure it would be impossible to arrange for examples similar to those of the 
previous section. 

However the situation is subtle, and one should remark that it is known ( |H.Tj p. 376, 
|Ka|IKV| ) that there are measurable sets of finite measure on R of which the Fourier Transform 
of their characteristic function vanishes in a prescribed interval. These sets are of the form 

E= \J[k-\ k ,k + \ k ]. 



with < Afc < 1/2 and (A&) € I . We will consider slightly more general sets namely open 
sets E = UfcGz( a fc' A)' with °<-k < Pk < a k+i, such that their complement can be written as 
the union of closed intervals whose length is bounded below. We call such sets open sets with 
lower bounded gaps. 

We will now show that such sets are characterized up to translations by there 3-deck. For 
this, let us first note that such sets admit the following characterization: 

Lemma 2.8. Let E CR. The following are equivalent: 
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(1) there exists an e > such that, for every x,y G E with \x — y\ < e, the interval [x, y] 
is contained in E; 

(2) up to a set of measure 0, E is an open set with lower bounded gaps. 

Proof. That (2) implies (1) is obvious. Let us assume that (1) holds and let Z be set of points 
in E that cannot be approximated either from the left or from the right by points of E. The 
set Z is countable as we can assign disjoint open intervals to its points. 

Now, let x G E \ Z. By construction of Z, there exists y G E such that x < y < x + e. 
Property (1) then implies that [x, y] C E. Similarly there is z G E with x — e < z < x and 
[z, x] C E. It follows that x G]z, y[c E and E \ Z is open. 

It is then obvious that E has gaps bounded bellow by e. □ 

We are now in position to prove the following: 

Theorem 2.9. Let E be an open set of finite measure with lower bounded gaps. Assume that 
the set F has the same 3-deck as E. Then F is a translate of E. 

Proof. Let us first prove that, up to a set of measure 0, F is also an open set with lower 
bounded gaps. For this, observe that Property (1) of Lemma 12.81 holds if and only if there is 
e > such that 

Ge(x, y) := / XE{t)X"E{t + x)XE(t + y)dt = 0, whenever < x < y < e. 

Further GE(x,y) = NE(0,y) — NE(x,y) and, as Np = Ne, it follows that F is also an open 
set with lower bounded gaps (up to measure 0). 

As above, write E = (a^,^) with < (3k < Ofc+i an d Te '■= inf(otk+i — (3k) > be 

the size of the smallest gap of E. Similarly, let be the size of the smallest gap of F. 

Note that NE(x,y) = f*g(—x) where f(t) = XE(t)xE(t + y) and g(t) = Xe(— It follows 
that d x NE(x,y) = —f * <?'(— x). Further, 

fcez 

where 5 X denotes a unit point mass at x G R, so that 

d x N E (x,y) = f * 6„/3 k (-x) - f*5- ak (-x) 
fcez 

= X] f( ak ~ x ) - f(Pk - x) 
fcez 

(2.6) = ^2 XE(ak ~ x)xE{otk - x + y) - XE((3k - x)xE((3k - x + y) 

fcez 

(the convergence of this series will soon be obvious in the case of interest to us). Now, if 
—Te < x < 0, then (3k < (3k — x < ak+i so that XE((3k — x) = 0. Further, as ak < ctk — x < 
Pk+^E < Ofc+ij XE(ctk-x) = 1 only when (3k — otk > x. As E is of finite measure, (3k — otk — ► 
so that the sum (|2.6j) is finite and 

(2.7) d x N E (x,y)= XE(a k -x + y). 
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Fix x small enough to have x < min(T£, Tp) and let Dp (resp. Dp) be the Fourier transform 
in the y variable of d x N E (x,y) (resp. of d x Np(x,y)). Then De is of the form D E {^) = 
Xe{0<p(£,) where = Ylk&z/ /3 k -a k >x e 2ln( - ak ~ x ^ is an analytic function, that is not iden- 
tically if x is small enough and such that </?(— £) = </?(£). Further, Dp is of the same form 

As we assumed Np = Np, we also have d x Np = d x Np and consequently Dp = Dp so that 
D E (Z)D E (ri)D E (-Z - 77) = Dp(OD F (ri)Dp(-Z - 77). 
Using the particular form of Dp and Dp, this is 

xe(.Oxe{v)xe(-^ - t/MCMt/M-C ~v) = xe(Oxe{v)xe{-£, - v)^{0^{v)^{-^ - v), 

for all £, 77 G R. But 

XiHOxiO/)xiH-£ - f ?) = xf(Oxf(v)xf(-^ - v) 

and is not zero if £, 77 are in some neighborhood of so that 

(2.8) viZMvM-t -v) = MMvM-Z - v) 

in that neighborhood. By analyticity of (p, if), (|2.8j) is valid for every £, 77 G R. It is then easy 
to see that the function 7 = t/?/<^ is of modulus 1 on the real line and satisfies 7(£ + 77) = 
7(^)7(77) (excepted for £,77 in some discrete set) so that 7(£) = e 2 * 7 ™^ for some a G R and 
VKO = e 2 "™^ (see Remark 3 after the proof of Lemma l2.3j) . 

Returning to D E = Dp, we get S(fM£H 0(0^(0^ e 2i7ra «x}?(£M6- As is 
analytic and not identically it follows that Xe(Q = e2l7ra ^XF(£) almost everywhere and F 
is a translate of E. □ 

3. The £>deck problem on Z/nZ 

3.1. Introduction. The aim of this section is essentially to bring together the knowledge 
about the problem in the combinatorics and crystallography communities. After summarizing 
known results and open questions, we will proceed with our own contributions. 

In order to do so, let us start with some notations: k, n will be two positive integers, 
k > 2 and £ = e 2m ^ n . We will not distinguish between sequences of n elements, n-periodic 
sequences and functions on TLjnL. If / = (/o, /1, . . . , /n-i) is such a function, its Fourier 
transform is defined by 

n-i 

7(0 = E/^' /GZ 
3=0 

with the natural extensions to higher dimension. The fe-deck of / is the function on (Z/nZ) fc_1 
given by 

fc-i 

Nf(ji, • • • Jk-i) = fsfi+h ■ ■ ■ fi+ik-i- 
3=0 

Its Fourier transform is then given by 

jvki, . . . , i k -t) = hh) . . . f(h-i)K-h - h-i). 
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Now let /, g be two functions on Z/nZ that take only non- negative values. As in the previous 
section, f,g have same /c-deck if and only if we can write g{k) = £(k)f(k) where £(k) is 
unimodular and such that 

(3.9) tth + ... + h-i)=Z(h)---Z(k-i) 

whenever l\, . . . , lk~i £ supp/ are such that l\ + . . . + l^-x G supp/. Note that ^ is a priori 
only defined on supp/. Our aim here is to know whether or not such a £ extends to a 
character of TLjnL in which case g is a translate of /. This is of course immediate if / does 
not cancel and k > 3. An other trivial case is when / cancels everywhere excepted at 0, that 
is / is constant. 

3.2. A quick overview of the 2-deck problem. Let us first concentrate on the case k = 2 

when condition (|3.9|) is void. We are seeking g such that |^| = |/|, which is a well known as 
the phase retrieval problem in crystallography. The study of this problem originates in the 
work of Patterson and a full solution has been given by Rosenblatt (see |Roj , |RoSj and the 
references therein). 

Let us briefly describe what may happen in the case / = xe and g = xf with E, F subsets 
of Z/nZ. One easily sees that E and F have same 2-deck if and only if E — E = F — F 
(counted with multiplicity). From this, one easily gets the trivial solutions F = E — a and 
F = —E — a. If this are the only solutions, we say that E is uniquely determined up to 
translations and reflections by its 2-deck. 

Note that not all sets are uniquely determined by there 2-deck. For instance, one may 
arrange for two sets A, B of cardinality at least 2 to be such that A — B and A + B (counted 
with multiplicity) are still sets {i.e. every element has multiplicity 1). One then immediately 
gets that E = A — B and F = A + B have same 2-deck, but as A and B have more than 2 
elements, F is not of the form E — a nor —E — a. Rosenblatt showed that this is almost always 
the case and gave a general solution to the problem, which is a bit to long to be summarized 
here. However, as he noticed himself, his solution may be difficult to use in practice. For 
instance, the following questions are open: 

Question 1 (RSI conjecture 2): Does the proportion of subsets ofL/nL that are not 
determined up to translation and reflection by their 2-deck go to as n goes to oo. 

Question 2: How many solutions (up to translations and reflection ) to the 2-deck problem 
can a subset of TLjnL of cardinality k have. 

This question is implicit in |E oj . 

3.3. Zeros in the spectrum of an indicator function. As noted in the introduction of 
this section, the fc-deck problem for k > 3 is trivially solved if x~E has no zeros. Before 
pursuing with our study we will therefore gather here some information about possible zeros 
of XE- 

First assume that n = pq (p, q not necessarily primes) and let / be n-periodic. Assume that 
/ is also p-periodic and write a = (/o, . . . , f p -i) for the period. We may see a as a function on 
L/pL and still write a for its Fourier transform on L/pL. An immediate computation shows 
that / is supported on the subgroup {0, q, . . . , (p — l)q} of L/nL and that f(lq) = qa{l). In 
particular, we get: 
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Fact 1. A subset E ofL/pL is uniquely determined up to translation by its k-deck, if and 
only if its q-periodic extension to Z/pqZ is uniquely determined up to translation by its 
k-deck. 

Notation : We will write qLjpL = {0, q, . . . , (p — l)q}. 

Recall that an n-th root of unity Q p , < p < n is said to be primitive if it is not an m-th 
root of unity for some m < n, that is, p and n are relatively prime. In particular, Q p is a 
primitive ^^y-th root of unity. The cyclotomic polynomial of order n is then defined as 

n (x-u)= n (*-c j ). 

u primitive ^ '■ U> n ) =1 } 

n-th roots 

It is well known that <l? n is the minimal polynomial of any primitive n-th root so that if P is 
a polynomial such that P(( l ) = then one has a factorization of P: P(x) = ^ n /u l n{x)Q{x), 
and PiC, 3 ) = whenever j and n/(n,l) are relatively prime. 

For instance, for E C Z/nZ, let Pe{x) = x J , so that Pe(( 1 ) = Xe(1)- This leads us to: 

jeE 

Fact 2. If xe(1) = for some I ^ 0, then xeU) = for all 1 < j < n with j and n/(n, I) 
relatively prime. 

Two particular cases are of interest to us: 

(1) if n = p a with p prime, then only the three following cases may occur: 

(a) xe(1) ± for all I, 

(b) Xe(1) = only for I of the form I = qp a ~ b , 1 < q < p b , 

(c) Xe(1) 7^ only for I of the form I = qp a ~ b , 1 < q < p b , in which case E is 
p 6 -periodic. 

(d) xe(1) = for all I ^ 0, in which case E = Z/nZ. 

(2) If n = pq with p, q two distinct primes, then only the five following cases may occur: 

(a) xe(1) ^ for all /, 

(b) Xe[1) 7^ unless I is either of the form I = jq, 1 < j < p, or of the form k = jp, 
1 < j < q, in which case xe(1) = f p + f q with f p p-periodic and f q g-periodic. 

(c) xe(1) = unless I = jq, 1 < j < p, in which case E is g-periodic. 

(d) xe(1) = unless / = jp, 1 < j < q, in which case E is p-periodic. 

(e) xe(1) = for all I ^ 0, in which case £ = TLjnL. 

3.4. Known results on Z/nZ. Let us now turn to the A:-deck problem. Most questions that 
arise naturally have been solved either in jRSlj or in |(tM| . More precisely, let us summarize 
what we consider as the main facts: 

(1) If n is a prime number, then every set (and even every integer valued function) is 
uniquely determined up to translations by its 3-deck; |E,Slj Theorem 3. The proof in 
|E,Slj is rather involved and we will give a simpler one bellow by a more explicit use 
of Fourier analysis. 

(2) Assume n is odd and E C Z/nZ. Let a = xe then if a(l) ^ 0, a is determined up to 
translations by its 3-deck; |GM| . Theorem 2. 



12 



PHILIPPE JAMING & MIHAIL N. KOLOUNTZAKIS 



(3) Every subset of Z/nZ is uniquely determined up to translation by its 4-deck; |GM| 
Theorem 5. This is further extended to integer valued functions in [DM Theorem 4: 
they are still determined up to translation by the 4-deck if n is odd but the 6-deck is 
needed when n is even. Moreover, examples are given there to show that these results 
are optimal. This disproves Conjecture 1 of jKolj. These examples show that as soon 
as n has 3 factors, n = pqr with p ^ q prime, then there are sets in Z/rzZ that are 
not uniquely determined up to translations by there 3-deck. 

(4) The proportion of subsets of TLjnL that are not uniquely determined up to translations 
by there 3-deck goes to as n — ► oo; |E,Slj Theorem 4. This is done by proving that 
the proportion of subsets E of Z/nZ such that x~e has a zero is at most 0(n -1 / 2+e ). 

3.5. Some new results. We will now settle the two remaining cases in which the 3-deck 
may suffice: n = p a with p prime and n = pq with p ^ q prime. 

Theorem 3.1. Let p be a prime number, p > 3 and let n = p a . Then every subset E off 
Z/nZ is uniquely determined up to translations by its 3-deck. 

Proof. Let us first prove the case n = p a prime. In this case, the particular cases of Fact 2 
imply that either: 

— for all Z, xe(1) ^ 0, or 

- E = Z/nZ. 

In both cases, E is uniquely determined up to translations by its 3-deck. This simplifies the 
proof in RSI . 

Assume now that p > 3 and that we have proved that every subset of Z/p a Z is uniquely 
determined up to translations by its 3-deck and let E C Z/p a+1 Z. Again, there are only four 
cases that may happen: 

— for all Z, Xe{1) ^ 0, or 

- E = Z/nZ, or 

— E is p b -periodic for some 1 < b < a, or 

- Xe(J) = only for I = qp b , 1 < q < p a ~ b . 

The two first cases are trivial whereas the third case is settled using the induction hypothesis 
and Fact 1. Let us now assume we are in the last case. 

Assume that g has same 3-deck as xe and write g(l) = £(1)xe(1) where £ satisfies 

(3.10) + l 2 ) = i{h)i{h) for all h,l 2 G supp£e such that h + 1 2 G suppxe. 

As p > 3, our assumption on E implies ^ and xi(2) ^ 0, so that (|3.1()j) implies 

£(2)=£(l + l)=£(i) 2 . _ 

Next, assume that we have proved that £(Z) = £(1)' for all I G suppxi; H [0, Iq]- We are 
then in one of the following cases: 

-I ,I + le suppxs and then (l3~T0l) implies f(Z + 1) = = by the 

induction hypothesis. 

— Iq + 1 ^ suppxi; and there is nothing to prove. 

- Iq ^ suppxe but Iq + 1 G suppxfi. But then Iq = qp b for some 1 < q < p a ~ b and as 
P > 3, /o — 1 is not of that form and so Zo — 1 G suppx^. But then (|3.1Uf) implies 
C(Z + l) = ^o-l)C(2)=C(l) /o+1 . 
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We conclude that £ is a character of Z/nZ and that g is a translate of xe- D 

Remark : The case p = 2 can be settled similarly by using the 4-deck instead of the 3-deck. 
One may then write £(fco + 1) = £(^o — 1)^(1)^(1), but this is not stronger then the result in 
jUMj . 

Note that the proof also holds for rational- valued functions on Z/nZ. 

Theorem 3.2. Lei n = pq with p,q two distinct primes. Then every subset of Z/nZ is 
uniquely determined up to translations by its 3-deck. 

Proof. In the case n = pq, four cases may happen: 

- for all I, x~e(1) 0) or 

— E = Z/nZ, or 

— E is p or -periodic, or 

- xe(1) = if and only if I is neither a multiple of p nor a multiple of g. 

The two first cases are again trivial whereas the third one is again settled using Fact 1 and 
the case n prime. 

Let us now exclude the last case using the fact that xe = Xe * Xe- 

Define f p , f q by 



( i 

f P (k) = 



2^(0) if/ = 

Xe(1) i/0a multiple of p , f q (l) 
else 



( \xe{$) in = o 

XeQ>) i/Oa multiple of q 
else 



so that xe = fp + f q . By assumption, f p (resp. / 9 ) has support {0, q, . . . , (p — l)q} (resp. 
{0,p, . . . , (q - l)p}). But = xe * Xe > that is f p *fp + f q *f q + 2f p * f q = f p + f q . As/ P */ P 
is supported in qZ/pZ + qZ/pZ = qZ/pZ, f q * / 9 is supported in pZ/qZ, we get that / p * / g 
has to be supported in qZ/pZL)pZ/qZ. Now let j G Z/nZ be outside this set. There exists a 
unique Zo £ {0, . . . ,p — 1} and a unique jo G {0, . . . , g — 1} such that j = £o<? + joP- Further 

p-1 

= f P * f q (j) = ^ f P (lq)f g (j ~ k) = fp(hq)Uhp) 
1=0 

so that one of f p (loq) fqQ>oP) 1S zero. This contradicts the assumption on the support of these 
functions. □ 

Remark : In the case of a rational valued function /, again the same four cases are to be 
considered and only the last one causes problems. If g has same 3-deck as /, we may write 
f = fp + fq and g = g p + g q with f p , f q , g p , g q defined as above replacing xe by / or g. 

It follows that f p , g p (resp. f q , g q ) are p-per iodic (resp. g-periodic) functions, their Fourier 
transforms over Z/pZ (resp. Z/qZ) don't vanish and they have same 3-deck. Therefore g p , 
(resp. g q ) is a translate of f p , (resp. f q ) on Z/pZ (resp. Z/qZ): 

g p (l) = e ^«/Mj(i) and g q (l) = f q (l) 

so that g(x) = f p (x — j) + f q {x — I) and this are the only possible solutions of the 3-deck 
problem for /. 



11 



PHILIPPE JAMING & MIHAIL N. KOLOUNTZAKIS 



Let us finally note that Grunbaum and Moore proved that one can not do any better. 
Indeed, if n = pqr with p, q two distinct primes and r > 3, let A = qrZ/pZ and B = prZ/qZ. 
Let E = A U (B + 1) and F = A U (B + 2), so that E and F are not translates of each 
other. Further, xe{x) = Xa{x) + Xb(x - 1), Xf = Xa(x) + Xb(x - 2) so that xe(Z) = 
+ e 2 ^/^(/) and ^(/) = + e ^/ n ^{l). 

Finally, as xa is supported in qr'L/p'L while xb is supported in prZ/qZ, it is easy to see 
that E and F have same 3-deck. 

Note that if r is also prime, this are essentially the only sets that have same 3-deck but 
are not translates of each other. 
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Appendix A. Continuity properties of Nf 
Let k be an integer and for fo, ■ ■ ■ , fk be simple functions on R, set xo = and define 

N f0! ... Jk (xi,...,x k ) = J f[fj(t-Xj)dt. 

3=0 

If fo = ■ ■ ■ = fk, we simply write Nf for A/,...,/. 
Lemma A.l. Let pi, . . . ,p k > 1. Then 

k 

\\ N fo,-,.fk\\L r (R k ) - EE Wfj\\L p J{R) 
j=0 

k 1 

holds for every measurable step functions on R z/ and only if 1 H — = > — . 

r p,- 
i=o 

// this last condition is satisfied, we may define Nf 0t .„j k for fj € U'i> by a usual approxi- 
mation process. 

Proof. Replacing fj(t) by fj(Xt) and letting A go to and to +oo shows that the condition 

k 

> 1 

is necessary. 



n k V- 1 • 
1 + - = > —is 
r * — ' Pi 

fc ^ 1 

For the converse, assume that 1 H — = > — and note that as not all p,-'s are 1, r > 1. 

Let (pi,...(pk G L r '(R) with - + — = 1. By density of functions of the form (pAxi) . . . <p k (x k ) 

r r / 

in U (R) and by duality, it is enough to prove that 

/ ipxfa) ■ ■ ■ (p k (x k )N fo ^. Jk {xi, . . . ,x k )dxi ■ ■ ■ ,dx k < Hvilla/TTll/jllp,-- 
But, by Fubini 

/ ip 1 (x 1 )---<p k (x k )Nf 0j _j k (xi,...,x k )dxi-- - ,dx k 

= Vi(ari) • • • ¥k(xk)fo(t)fi(t - xi) ■ ■ ■ f k (t - x k )dtdxi ■ ■ ■ dx k 

JR JR k 

= fo(t) / <fi(x 1 )f 1 (t-x 1 )dx 1 ---<p k (x k )f 1 (t-x k )dx k 
(A.ll) = ( fo(t)ifi * h(t) ■■■<p k * f k (t)dt, 

JR 

with Now, define p 7 - by 1 + — = — + -7 = — + 1 — - and note that 

pj pj r Pj r 

11 111 lk 
— + — + h — = — + — + ••• + = 1, 

Po Pi Pk Po Pi Pk r 
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so that Holder's inequality implies that (|A.11|) is 

¥>i * fi 



< 11/olU 



pi 



* fk 



To conclude, as 1 + J- = — + h, Young's inequality implies that this is 

Pi Pi ~ 

as wanted. □ 
Lemma A. 2. Let f be a non-negative L 1 -function, then 

lim N f (x 1 ,...,x k ) = [ f k dt. 

Proof. For sake of simplicity, we will only prove this for k = 2. So, let 

N f (x,y)= [ f{t)f(t-x)f(t-y)dt. 

JR 

By Holder's inequality, 

N f (x,y) < [ f(t)dt 



so that we only have to prove liminfiVj > J R f 3 (t)dt. For this, let us consider the cake 
representation of /: 



/(*) = / X{f>u}(t)du, 

and write (with Fubini) 



OO ^00 /"OO 



N f^ x ^y) = LJ J J X{f>u}( t )X{f>v}(t - x)x{f >w }(t - y)dudvdwdt 



oo poo poo 



N x {f> u},x { f >v} ,x { f >w} ( x iy) dudvdw - 

It follows from Fatou's Lemma that it is enough to prove that 

liminf N XEtXFjXG (x,y) > N XE:XFtXG (0,0), 

x,y— >U 

for every measurable sets E, F, G of finite measure. 

Now, fix 5 > 0, by regularity of the Lebesgue measure, there exists open sets U, V such 
that F C U, \U \ F\ < 5 and G C V, \V \ G\ < S. Then 



N xe,xf,xg( x 'V) = / XE(t)xF(t - x)xc(t - y)dt 

JR 

> [ XE(t)xu(t-x) Xv {t-y)dt-2S 



(A.12) 

> / XE(t)x F (t)xG(t)xu{t-x) X v(t-y)dt- 25. 

JR 

But, as U, V are open, 

XE(t)XF(t)XG{t)Xu{t ~ x)xv{t -y)-> XE(t)x F {t)XG(t)xu(t)xv{t) = XE{t)X F {t)XG{t) 
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when x,y -> and XE(t)xF(t)xG(t)xu{t ~ x)\v{t ~ y) < Xe 6 L X (R). By dominated 
convergence, we then have 

/ Xs(*)XF(*)XG(%l/(*-ic)xv(*-2/)d*-* / XE(t)XF(t)xG(t) = N XE:XF:XG (0,0). 

JR JR 

As 5 is arbitrary, we get the desired result from (|A.12j) . □ 
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